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Synchronizing Watermark Detectors in Geometrically Distorted 

Signals 

Technical Field 

The invention relates to steganography, data hiding, and digital watermarking. 

5 Background and Summary 

~: Digital watermarking is a process for modifying physical or electronic media to 

S embed a machine-readable code into the media. The media may be modified such that 

hi the embedded code is imperceptible or nearly imperceptible to the user, yet may be 

% detected through an automated detection process. Most commonly, digital watermarking 

W 10 is applied to media signals such as images, audio signals, and video signals. However, it 
y : may also be applied to other types of media objects, including documents (e.g., through 

* ! line, word or character shifting), software, multi-dimensional graphics models, and 

In surface textures of obj ect s . 

y s Digital watermarking systems typically have two primary components: an 

15 encoder that embeds the watermark in a host media signal, and a decoder that detects and 
reads the embedded watermark from a signal suspected of containing a watermark (a 
suspect signal). The encoder embeds a watermark by altering the host media signal. The 
reading component analyzes a suspect signal to detect whether a watermark is present. In 
applications where the watermark encodes information, the reader extracts this 
20 information from the detected watermark. 

Several particular watermarking techniques have been developed. The reader is 
presumed to be familiar with the literature in this field. Particular techniques for 
embedding and detecting imperceptible watermarks in media signals are detailed in the 
assignee's co-pending application serial number 09/503,881 and US Patent 6,122,403, 
25 which are hereby incorporated by reference. 

One particular problem in digital watermarking applications is synchronizing a 
detector to deal with geometric warping distortion of a watermarked image. A number of 
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techniques have been developed for dealing with geometric distortion in watermarked 
images. One technique is to make the watermark more robust to geometric distortion by 
embedding it in attributes of the image that are relatively invariant to geometric 
distortion. While this improves detection in some cases, it typically does not address all 
5 forms of geometric distortion and more complex, non-linear geometric distortion. 

Another technique is to include geometric calibration features in the watermark signal 
that enable detection and estimation of the geometric distortion parameters, such as 
rotation and scale. These features include, for example, calibration signal peaks in a 
particular transform domain, such as the autocorrelation domain and/or Fourier domain. 

1 0 These techniques use correlation techniques or other pattern matching methods to 

estimate affme geometric distortion parameters. For example, cross correlation of the 
received signal and the calibration signal in particular transform domains produce 
correlation peaks that correspond to affme distortion parameters, such as rotation, scale, 
translation, shear, and differential scale. 

15 Such techniques do not provide error free estimation of geometric distortion. In 

some cases, errors result from estimation, even where the geometric distortion is linear. 
In addition, affine transform parameters can only approximate geometric distortion that is 
non-linear. For example, it can provide a piecewise linear estimate of affine distortion in 
sub-blocks of an image, but it cannot always accurately represent non-linear distortion in 

20 a manner that leads to error free recovery of the digital watermark message. 

Similar problems occur in audio watermarking where temporal distortions, such 
as pitch invariant time scale changes, linear speed changes, up/down sampling, cropping, 
and lossy compression, make it difficult to synchronize the detector for accurate decoding 
of embedded message payloads. These and other distortions introduce a form of 

25 geometric distortion of the digital watermark signal in the host audio signal. 

The invention provides a method for synchronizing a watermark detector. One 
aspect of the invention is a method of synchronizing a digital watermark detector. The 
detector operates on a watermarked signal where a watermark signal has been distributed 
throughout the host media signal, and preferably, repeatedly embedded in segments of the 
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host media signal (e.g., music, image or video signal). The method divides a 
watermarked signal into blocks, each block including a portion of a watermark signal. 
For example, in the case where the watermark is replicated in segments of the host signal, 
these blocks sub-divide each segment into smaller parts. For each block, the method 
5 computes a local correlation space comprising a neighborhood of correlation values by 
correlating the watermarked data in the block with a known watermark signal at a 
neighborhood around the block. It then finds a correlation maxima in the local 
correlation space for each block, where the correlation maxima indicates a local offset 
used to align the watermarked data in the block before decoding a watermark message 

10 from the block. 

This method applies to watermarked signals of varying media types. In addition, 
it may be used to synchronize watermark detection and message reading in various signal 
domains, such as a spatial domain, temporal domain, frequency domain, or some other 
transform domain (e.g., autocorrelation, wavelet, Fourier, etc.). For example, it can be 

1 5 applied in the spatial domain or spatial frequency domain for images and video frames, as 
well as the time or frequency domain for audio. It may also be applied in the time- 
frequency domain for audio and video signals. 

In one implementation, the method uses an estimate of affine geometric distortion 
parameters to transform an image block in the watermarked image to a position 

20 approximating an original orientation of the image block in the watermarked image. It 
then shifts the transformed image block to neighboring locations. The method then 
computes a correlation surface by finding the correlation between the watermark signal 
and the transformed block at its location and each of the neighboring locations. The 
method finds a correlation maximum in the correlation surface formed by the correlation 

25 values in the neighborhood. The location of the correlation maximum provides an offset 
value that further refines the orientation of the image data. A message decoder then 
decodes a watermark message from the watermarked image adjusted by the offset value. 

Further features will become apparent with reference to the following detailed 
description and accompanying drawings. 
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Brief Description of the Drawings 

Fig. 1 is a flow diagram of a method for synchronizing a watermark detector. 

Detailed Description 

5 Fig. 1 is a flow diagram of a method for synchronizing a watermark detector. The 

method operates on a watermarked image 100 that has been geometrically distorted, 
either intentionally or unintentionally. Before the method begins, a global 
O synchronization method, such as those described in watermarking literature is used to 

j7; compute an estimate of the affine geometric distortion of the image since being 

w 1 0 embedded with the digital watermark. 

w 

fin The method operates on small blocks of pixels within an image frame. To extract 

a watermark message from the block, a watermark reader first needs to determine the 
geometric distortion and compensate for it. The method described here attempts to 

fl; provide a more accurate measure of the geometric distortion by refining the estimate of 

pi 15 affine geometric distortion. 

^ : To accomplish this, the method applies the affine transform to the small block 

(102) using the estimated affine transform parameters (104). The method interpolates the 
image samples computed from the affine transform to compute image sample values at 
discrete sample locations within the transformed block. These interpolated sample values 

20 form a block of image data approximating a block in the watermarked image at the time 
of embedding. However, errors due to estimation and non-linear distortion remain. 

Next, the method computes the local correlation between the known component 
of the watermark signal and the interpolated block (108). The method then shifts the 
interpolated block by one discrete sample location to each of the 8 neighboring locations 

25 (110). For each of these locations, it repeats the correlation computation. The correlation 
is computed as the inner product of the interpolated/shifted version of the watermarked 
image block and the watermark signal block expected for that location based on the affine 
transform parameters. In total, the correlation process produces a two dimensional array 
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(e.g., 3 by 3) of correlation values, forming a correlation surface from the block. Since 
the interpolated and corresponding shifted blocks are comprised of image samples at 
integer locations, the correlation surface is also comprised of correlation values at integer 
locations. 

5 The method then estimates the sub-sample location of the correlation maximum in 

the correlation surface. While there are alternative ways to compute this sub-sample 
location, our current implementation uses a center of mass approach. The center of mass 
location identifies a sub-sample offset location and corresponding offset value that may 
h be used to align the data before decoding the watermark message symbols embedded in 

"r\ 10 it. If the watermark signal is present with sufficient energy based on the correlation value 
LU and the offset exceeds a threshold value, it is used to realign the image data in the block 

before watermark decoding. 

The implementation repeats the above process on small blocks across the 
watermarked image. In our implementation, the complete watermark message is 
15 replicated in larger blocks throughout the image, and the small blocks are sub-blocks of 
the larger block. As such, the entire message can be extracted from a subset of the image 
using a collection of the smaller blocks. 

By repeating the above process for rows and/or columns of sub-blocks, the 
watermark detector generates an array of sub-sample offsets, each providing an estimate 
20 of coordinates used to align local blocks of the watermarked data before watermark 
message extraction. A further enhancement of the method stores these sub-sample 
offsets and applies curve fitting to fit the offsets (e.g., the horizontal and vertical 
translational shift vectors) to a curve. This curve fitting process filters the array of 
offsets to provide a refined array of sub-sample offsets. This refined set of offsets is then 
25 used to align the watermark image data before performing message decoding operations. 

A further enhancement is to use the offsets and the correlation values associated 
with each one to get more accurate alignment coordinates. In particular, the curve fitting 
model weights the offsets by their corresponding correlation value so that offsets with 
lower correlation values are given less weight. Further, in locations where the correlation 
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values are low or the offset values appear noisy, the watermark detector can target 
message decoding operations away from these locations, and instead, focus on blocks of 
data that have higher correlation values and/or less noisy offset values. To get more 
accurate message decoding, the watermark reader selectively decodes embedded message 
5 symbols from sub-blocks within a block carrying a watermark signal, where the sub- 
blocks have higher detection metrics as indicated by the correlation values and array of 
offsets of neighboring sub-blocks. This enables message decoding operations to be 
focused on localized regions within a block of the received signal 
q While the message decoding operations vary depending on the nuances of the 

!H 1 0 watermark embedding method, our implementation decodes message symbols as follows. 
W It begins by predicting the values of chips in a spread spectrum signal from the re-aligned 

ffi data. Multiple chips map to each embedded error correction encoded message element. 

w For each error correction encoded message signal element, the chips are aggregated to 

H produce a soft value, which is input to a Viterbi decoder. The Viterbi decoder performs 

Sj 15 error correction decoding to recover message symbol elements. 
JL 1 Other forms of message decoding may be used depending on the nature of the 

M embedding methodology, such as mapping image attribute values into quantization bins, 

correlation based message symbol decoding, etc. 

There are a number of potential variations and enhancements to the digital 
20 watermark detection method described above. The method refines the synchronization of 
the watermark signal by computing a number of local correlations on blocks throughout 
the received signal. For each localized correlation around a block of watermarked data, 
the method computes a local correlation space comprising an array of correlation values 
by correlating a known portion of the watermark signal with the block of watermarked 
25 data. It then finds the maximum correlation peak in this local correlation space. The 
process of finding the correlation peak can include a variety of techniques, such as the 
center of mass calculation described above, as well as a median calculation. Another way 
to locate the peak is to apply an interpolation filter followed by a reconstruction filter to 
the local correlation space. For example, in an image, one way to locate the peak is to 
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first up-sample the local correlation space, and then find the sub-pixel point with 
maximum correlation as the offset estimate. 

The method provides an efficient way to synchronize watermark detection at local 
points in the watermarked signal because it avoids repeated interpolation calculations to 
5 find the appropriate local reference coordinate for reading embedded symbols of the 

watermark message (e.g., x-y coordinates or time coordinates used to align the watermark 
reader on a block of data). The method can be applied to small sub-blocks of data, such 
as a 32 by 32 or 16 by 16 block of pixels, even if the entire watermark signal is spread 
p over a larger area. By reducing the size of the local correlation space, the method can 

rf: 10 provide refined synchronization offsets over a smaller area. However, the size of the 
W block needs to be large enough to provide meaningful correlation values, 

rfi The above method applies to a variety of media types, like video and audio. In 

w audio, the synchronization technique may be applied to time segments within the 

H : watermarked audio signal. For example, consider a case where a watermark signal is 

Hi 15 replicated in blocks of the audio signal. The method computes local correlation spaces 

around sub-blocks of the audio blocks including a portion of a watermark signal. It then 
H- finds the correlation maxima in the local correlation space, and the coordinates of the 

maxima align the message reader to decode embedded message symbols that form part of 
the watermark signal in that sub-block. 
20 The above method applies to data in different domains, such as the spatial domain 

or spatial frequency domain for still and moving images, and time or frequency domain 
for audio signals. It may also be applied to the time-frequency domain for audio and 
video signals. For example, it can be used in the synchronization of a time-frequency 
domain watermark for audio or video signals, such as described in co-pending U.S. 
25 Patent Application No. 09/661 ,900, which is hereby incorporated by reference. 

The method enables improved detection in cases where the watermarked signal is 
distorted by one or more processes, such as geometric distortion, time scale distortion, 
distortion due to lossy compression, Analog to Digital or Digital to Analog conversion, 
etc. 
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Concluding Remarks 

Having described and illustrated the principles of the technology with reference to 
specific implementations, it will be recognized that the technology can be implemented in 
many other, different, forms. To provide a comprehensive disclosure without unduly 
5 lengthening the specification, applicants incorporate by reference the patents and patent 
applications referenced above. 

The methods, processes, and systems described above may be implemented in 
hardware, software or a combination of hardware and software. For example, the 
auxiliary data encoding processes may be implemented in a programmable computer or a 

10 special purpose digital circuit. Similarly, auxiliary data decoding may be implemented in 
software, firmware, hardware, or combinations of software, firmware and hardware. The 
methods and processes described above may be implemented in programs executed from 
a system's memory (a computer readable medium, such as an electronic, optical or 
magnetic storage device). 

1 5 The particular combinations of elements and features in the above-detailed 

embodiments are exemplary only; the interchanging and substitution of these teachings 
with other teachings in this and the incorporated-by-reference patents/applications are 
also contemplated. 



